The present invention relates to an artificial neuron element for neural networks, and more particularly to an artificial neuron element with electrically programmable synaptic weight.
Artificial neural network models have been studied for many years in the hope of achieving human-like performance in the fields of speech and image recognition. An artificial neural network is a massively parallel array of simple computational elements (neurons) that models some of the functionality of the human nervous system and attempts to capture some of its computational synaptic strengths or weights. The abilities that an artificial neural netmight aspire to mimic include the ability to consider many solutions simultaneously, the ability to work with corrupted or incomplete data without explicit error correction, and a natural fault tolerance.
Neural network implementations fall into two broad categories, digital and analog. Digital neural networks and analog neural networks have the following strengths and weaknesses respectively, as described in the literature "Asynchronous VLSI Neural Networks Using Pulse-Stream Arithmetic", A. F. Murray and A. V. W. Smith, IEEE Journal of Solid-State Circuits, Vol. 23, No. 3, pp. 688-697, 1988. The strengths of a digital approach are:
Design techniques are advanced, automated, and well-understood; PA1 Noise immunity is high; PA1 Computational speed can be very high; and PA1 Learning networks can be implemented readily. PA1 Digital circuits of this complexity must be synchronous, while real neural nets are asynchronous; PA1 All states, activities, etc. in a digital network are quantized; and PA1 Digital multipliers, essential to the neural weighting function, occupy a large silicon area. PA1 Asynchronous behavior is automatic; PA1 Smooth neural activation is automatic; and PA1 Circuit elements can be small. PA1 Noise immunity is slow; PA1 Arbitrarily high precision is not possible; and PA1 Worst of all, no reliable analog, nonvolatile memory technology exists.
However, for digital neural networks, there are several unattractive features:
The benefits of analog networks are:
Drawbacks to analog neural networks include:
Biological neural nets, by their nature, are nonlinear, and are typically analog. At present, however, the rich properties of neural networks associated with massively parallel processing using analog neurons and synapses have not been fully explored. To make neural computing hardware more powerful, compact and electrically programmable synapses are needed. By using programmable neural chips with weight-adjustable neurons and adaptive synapses, reconfigurable neural systems with learning capabilities can be constructed.
Therefore, the present invention is directed toward the development of a viable method of storing neural network weights or network connection strengths in analog form by using digital circuit technology to facilitate the integrated circuit design: